TRAINING OBJECTIVE

On the Hidden Objective Biases of Group-based Reinforcement Learning (2026.acl-short)

Challenge:	Existing studies have used pre-trained language models for domain adaptation or transfer learning to improve natural language processing performance.
Approach:	They propose to fine-tune word sense disambiguation on sequence-pair ranking task and to use existing WordNet examples to augment the model.
Outcome:	The proposed model achieves state-of-the-art on the English all-words benchmark datasets.

Copied to clipboard

Aleksandar Fontana, Marco Simoni, Giulio Rossolini, Paolo Mori, Andrea Saracino

Challenge:	Recent studies have reported unexpected behaviors during training, including lengthrelated biases, formatting tokens, and reward hacking in multi-objective settings.
Approach:	They propose to analyze group-based reinforcement learning methods within a unified surrogate formulation.
Outcome:	The proposed methods exhibit structural mismatches between reward optimization and the underlying training objective.

Permutative Preference Alignment from Listwise Ranking of Human Judgments (2025.emnlp-main)

Copied to clipboard

Yang Zhao, Yixin Wang, Mingzhang Yin

Challenge:	Existing methods to align Large Language Models with human preferences are based on the Bradley-Terry model, but when multiple responses are available, the B-T model fails to guarantee an accurate list ranking of the responses.
Approach:	They propose an offline listwise approach that incorporates the Normalized Discounted Cumulative Gain (NDCG) as an alternative training objective for LLM alignment.
Outcome:	The proposed approach outperforms existing pairwise and listwise methods on evaluation sets and general benchmarks such as AlpacaEval.

MR-P: A Parallel Decoding Algorithm for Iterative Refinement Non-Autoregressive Translation (2022.findings-acl)

Copied to clipboard

Hao Cheng, Zhihua Zhang

Train Once, and Decode As You Like (2020.coling-main)

Challenge:	Non-autoregressive neural machine translation models remove dependency between tokens in the target sentence and generate all tokens on parallel .
Approach:	They propose a non-autoregressive neural machine translation model that decodes with the Mask-Predict algorithm which iteratively refines the output.
Outcome:	The proposed algorithm increases the performance of the WMT’14 translation task by 1.39 points.

Copied to clipboard

Chao Tian, Yifei Wang, Hao Cheng, Yijiang Lian, Zhihua Zhang

HydraSum: Disentangling Style Features in Text Summarization with Multi-Decoder Models (2022.emnlp-main)

Challenge:	Existing approaches to machine translation support autoregressive, semi-autoregressive and refinement-based non-auto-regressives.
Approach:	They propose a unified approach for supporting different generation manners of machine translation including autoregressive, semi-autoregressive and refinement-based non-auto-regressives.
Outcome:	The proposed approach achieves better or competitive translation performance compared with strong baseline models in all the settings.

Copied to clipboard

Tanya Goyal, Nazneen Rajani, Wenhao Liu, Wojciech Kryscinski

Summarization

Challenge:	Abstractive summarization systems implicitly encode “decisions” about summary properties, but these are not enforced.
Approach:	They propose a new summarization architecture that extends existing models to a mixture-of-experts version with multiple decoders.
Outcome:	The proposed architecture outperforms baseline models in obtaining stylistically-diverse summaries by sampling from individual decoders or their mixtures.

Pruning Pre-trained Language Models Without Fine-Tuning (2023.acl-long)

Copied to clipboard

Ting Jiang, Deqing Wang, Fuzhen Zhuang, Ruobing Xie, Feng Xia

Hierarchical Curriculum Learning for AMR Parsing (2022.acl-short)

Challenge:	Existing methods to prune Pre-trained Language Models (PLMs) are overparameterized and require fine-tuning.
Approach:	They propose a pruning method that uses first-order pruning to prune PLMs while fine-tuning the remaining weights.
Outcome:	The proposed method outperforms first-order pruning and zero-order methods at sparsity levels.

Copied to clipboard

Peiyi Wang, Liang Chen, Tianyu Liu, Damai Dai, Yunbo Cao, Baobao Chang, Zhifang Sui

Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination (D18-1)

Challenge:	Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, but there is a gap between their flat training objective and the hierarchic structure, which limits the model generalization.
Approach:	They propose a Hierarchical Curriculum Learning framework with Structure-level (SC) and Instance-level curricula (IC) that aims to translate sentences to semantic representation with a hierarchical structure.
Outcome:	Experiments on AMR2.0, AMR3.0, structure-complex and out-of-distribution situations confirm the effectiveness of the proposed framework.

Copied to clipboard

Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, Jianqiang Zhao

Dice Loss for Data-imbalanced NLP Tasks (2020.acl-main)

Challenge:	Experimental results on Chinese-English and English-French multi-domain translation tasks demonstrate the effectiveness of the proposed model.
Approach:	They propose to use mixed-domain parallel sentences to construct a unified model that allows translation to switch between different domains.
Outcome:	The proposed model distinguishes and exploits word-level domain contexts on Chinese-English and English-French translation tasks.

Copied to clipboard

Xiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang, Fei Wu, Jiwei Li

Syntactic Text Processing

Challenge:	Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension .
Approach:	They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks.
Outcome:	The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks.

Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis (P19-1)

Copied to clipboard

Jialong Tang, Ziyao Lu, Jinsong Su, Yubin Ge, Linfeng Song, Le Sun, Jiebo Luo

Sentiment Analysis

Challenge:	Experimental results show that our proposed approach yields better attention mechanisms . dominant ASC models are mostly discriminative classifiers based on manual feature engineering .
Approach:	They propose a self-supervised approach to aspect-level sentiment classification that mines useful attention supervision information from a training corpus to refine attention mechanisms.
Outcome:	The proposed approach yields better attention mechanisms on multiple datasets.

Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection (2022.naacl-main)

Copied to clipboard

Angelica Chen, Vicky Zayats, Daniel Walker, Dirk Padfield

NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist (2023.acl-long)

Challenge:	a recent study shows that current NLP models operate non-incrementally, causing unacceptable delays for the user.
Approach:	They propose a streaming BERT-based sequence tagging model that detects disfluencies in real-time . they train the model to decide whether to immediately output a prediction or wait for further context .
Outcome:	The proposed model produces accurate predictions sooner than baselines, with lower flicker . disfluencies hurt readability of ASR transcripts, erode model performance on downstream tasks .

Copied to clipboard

Iftitahu Nimah, Meng Fang, Vlado Menkovski, Mykola Pechenizkiy

Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation (2024.acl-long)

Challenge:	a systematic review of automatic evaluation metrics for Natural Language Generation (NLG) shows that task-agnostic metrics have a weak correlation with human .
Approach:	They propose a framework to assess the effectiveness of automatic metrics in three NLG tasks . they propose task-agnostic and human-aligned metrics to be used for evaluation .
Outcome:	The proposed framework provides access to the evaluation tools for three NLG tasks.

Copied to clipboard

Jiaxin Bai, Yicheng Wang, Tianshi Zheng, Yue Guo, Xin Liu, Yangqiu Song

Improving QA Generalization by Concurrent Modeling of Multiple Biases (2020.findings-emnlp)

Challenge:	Abductive reasoning is the process of making educated guesses to provide explanations for observations.
Approach:	They propose a task of complex logical hypothesis generation to generate a complex logique hypothesis that can explain a set of observations.
Outcome:	The proposed model generates logical hypotheses closer to the reference hypothesis, but not better on unseen observations.

Copied to clipboard

Mingzhu Wu, Nafise Sadat Moosavi, Andreas Rücklé, Iryna Gurevych

Responsible & Trustworthy NLP

Challenge:	Existing approaches for debiasing datasets are weaker than current approaches for generalization.
Approach:	They propose a framework for analyzing multiple biases in training data to reduce bias weighting.
Outcome:	The proposed framework improves generalization on in-domain and out-of-domain datasets by weighting examples based on their strengths and bias strengths.

NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation (2023.findings-emnlp)

Copied to clipboard

Peter West, Ronan Bras, Taylor Sorensen, Bill Lin, Liwei Jiang, Ximing Lu, Khyathi Chandu, Jack Hessel, Ashutosh Baheti, Chandra Bhagavatula, Yejin Choi

Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models (2025.findings-naacl)

Challenge:	a new commonsense knowledge model, NovaCOMET, combines knowledge and general task models.
Approach:	They propose an open commonsense knowledge model that combines knowledge and general task models.
Outcome:	The proposed model matches or exceeds existing knowledge models on commonsense reasoning tasks.

Copied to clipboard

Aashiq Muhamed, Mona T. Diab, Virginia Smith

CNNBiF: CNN-based Bigram Features for Named Entity Recognition (2021.findings-emnlp)

Challenge:	Sparse Autoencoders (SAEs) are a promising tool for disentangling FM representations, but they struggle to capture rare, yet crucial concepts in the data.
Approach:	They propose a technique to train Sparse Autoencoders to illuminate elusive dark matter features by focusing on specific subdomains.
Outcome:	The proposed method achieves 12.5% better classification accuracy than general-purpose SAEs when applied to remove spurious gender information.

Copied to clipboard

Chul Sung, Vaibhava Goel, Etienne Marcheret, Steven Rennie, David Nahamoo

Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation (2021.acl-long)

Challenge:	Named entity recognition tasks require a self-attention mechanism with unconstrained length that fails to capture local dependencies.
Approach:	They propose a joint training objective which better captures the semantics of words corresponding to the same entity by augmenting the objective with a group-consistency loss component.
Outcome:	The proposed model achieves a test F1 of 93.98 with a single transformer model.

Copied to clipboard

Elena Voita, Rico Sennrich, Ivan Titov

Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction (2023.findings-acl)

Challenge:	Neural Machine Translation models are influenced by two types of context, source and target, but none explicitly evaluates relative contribution to generation decision.
Approach:	They propose to adopt a variant of Layerwise Relevance Propagation which evaluates relative contributions to the generation decision by a proportion of token influence.
Outcome:	The proposed model can evaluate the relative contribution of source and target to the generation decision by using a variant of Layerwise Relevance Propagation (LRP)

Copied to clipboard

Thong Nguyen, Xiaobao Wu, Xinshuai Dong, Cong-Duy Nguyen, Zhen Hai, Lidong Bing, Anh Tuan Luu

Visual Question Decomposition on Multimodal Large Language Models (2024.findings-emnlp)

Challenge:	Existing studies have shown that FCNNs perform inefficient splitting for review features, making it difficult to clearly differentiate helpful from unhelpful reviews.
Approach:	They propose a listwise attention network that captures the MRHP ranking context and a pairwise optimization objective that enhances model generalization.
Outcome:	The proposed framework achieves state-of-the-art results and polished generalization performance on two large-scale MRHP benchmark datasets.

Copied to clipboard

Haowei Zhang, Jianzhe Liu, Zhen Han, Shuo Chen, Bailan He, Volker Tresp, Zhiqiang Xu, Jindong Gu

Factorising Meaning and Form for Intent-Preserving Paraphrasing (2021.acl-long)

Challenge:	Existing methods for question decomposition focus on unimodal language models, but question decomposing capability of Multimodal Large Language Models (MLLMs) has yet to be explored.
Approach:	They propose a finetuning dataset and a training objective for selective decomposition to enhance the model's question decomposing capability.
Outcome:	The proposed dataset shows that existing models struggle to produce high-quality sub-questions.

Copied to clipboard

Tom Hosking, Mirella Lapata

BERTTune: Fine-Tuning Neural Machine Translation with BERTScore (2021.acl-short)

Challenge:	Existing methods for paraphrasing questions retain original intent but use a different surface form.
Approach:	They propose a method for generating paraphrases of English questions that retain original intent but use a different surface form.
Outcome:	The proposed method generates paraphrases with better tradeoff between semantic preservation and novelty compared to previous methods.

Copied to clipboard

Inigo Jauregi Unanue, Jacob Parnell, Massimo Piccardi

Fine-grained Post-training for Improving Retrieval-based Dialogue Systems (2021.naacl-main)

Challenge:	Neural machine translation models are biased toward limited translation references . BERTScore is a scoring function based on contextual embeddings that overcomes the limitations of n-gram-based metrics.
Approach:	They propose to fine-tune models with a new evaluation metric based on contextual embeddings to overcome the limitations of n-gram-based metrics.
Outcome:	The proposed training objective improves translations that are different from the translations but close in the contextual embedding space.

Copied to clipboard

Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko, Jungyun Seo

Natural Language Interfaces

Challenge:	Existing methods to select the correct response for a dialogue system are generation-based and retrieval-based.
Approach:	They propose a fine-grained post-training method that reflects the characteristics of the multi-turn dialogue.
Outcome:	The proposed model achieves state-of-the-art with significant margins on three benchmark datasets.

Translation-based Supervision for Policy Generation in Simultaneous Neural Machine Translation (2021.emnlp-main)

Copied to clipboard

Ashkan Alinejad, Hassan S. Shavarani, Anoop Sarkar

Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling (2022.naacl-main)

Challenge:	Existing approaches to train simultaneous machine translation agents have been used to find the optimal action sequences for translation quality and lag.
Approach:	They propose a supervised learning approach that detects minimum reads required for generating target tokens by comparing simultaneous translations against full-sentence translations.
Outcome:	The proposed method produces much higher quality translations while minimizing the average lag in simultaneous translation.

Copied to clipboard

Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Daxin Jiang

Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention (2020.coling-main)

Challenge:	Existing methods to train cross-lingual pre-trained language models have shown great success in cross-linguistic sequence labeling tasks.
Approach:	They propose a cross-lingual language informative span masking task to eliminate the objective gap between pre-training and fine-tuning stages.
Outcome:	The proposed method surpasses the state-of-the-art methods on multiple benchmarks even with limited pre-training data.

Copied to clipboard

Yang Li, Tao Shen, Guodong Long, Jing Jiang, Tianyi Zhou, Chengqi Zhang

Information Extraction & Text Mining

Challenge:	Existing approaches to handle wrong labeling and long-tail relations are labor-intensive and scarce training data.
Approach:	They propose a neural network to handle wrong labeling and long-tail relations by collaborating relation-augmented attention.
Outcome:	The proposed neural network improves the state-of-the-art on the NYT dataset .

Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation (2022.naacl-main)

Copied to clipboard

Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu

Bringing Real-World Relations into Video Generation with Graph-Structured Knowledge (2026.acl-long)

Challenge:	Neural text generation models are typically trained by maximizing log-likelihood with the sequence cross entropy (CE) loss.
Approach:	They propose an Edit-Invariant Sequence Loss method which computes the matching loss of a target sequence with all n-grams in the generated sequence.
Outcome:	The proposed method outperforms the common CE loss and strong baselines on a wide range of tasks.

Copied to clipboard

Joonhyung Park, Jaeyun Song, Sihwan Park, Eunho Yang

Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval (2021.acl-long)

Challenge:	Existing text-to-video models struggle to accurately simulate real-world physics and dynamic entity interactions.
Approach:	They propose a framework that integrates graph-structured temporal knowledge into video latent diffusion models to enhance compositional generation and interaction fidelity.
Outcome:	The proposed framework enhances compositional generation and interaction fidelity by integrating graph-structured temporal knowledge into video latent diffusion models.

Copied to clipboard

Zijing Ou, Qinliang Su, Jianxing Yu, Bang Liu, Jingwen Wang, Ruihui Zhao, Changyou Chen, Yefeng Zheng

Information Retrieval

Challenge:	Existing methods for document hashing combine only one of semantics and neighborhood information, lacking a theoretical principle to guide the integration process.
Approach:	They propose to encode neighborhood information with a graph-induced Gaussian distribution and integrate it with generative models.
Outcome:	The proposed model can be trained as efficiently as state-of-the-art methods on benchmark datasets.

Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation (2023.findings-emnlp)

Copied to clipboard

Gunsoo Han, Daejin Jo, Daniel Nam, Eunseop Yoon, Taehwan Kwon, Seungeun Rho, Kyoung-Woon On, Chang Yoo, Sungwoong Kim

Dialogue Systems & Conversational Agents

Challenge:	Existing knowledge-grounded dialogue generation algorithms require annotated knowledge to generate a response grounded on the retrieved knowledge.
Approach:	They propose an efficient algorithm for latent variable modeling that leverages large amount of dialogue data.
Outcome:	The proposed algorithm outperforms the supervised learning algorithm on knowledge-grounded dialogue datasets while maintaining efficiency and scalability.

Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models (2023.eacl-main)

Copied to clipboard

Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer

PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition (2023.findings-acl)

Challenge:	Pretrained language models store a large amount of factual information that can be elicited by prompting or finetuning.
Approach:	They propose methods to measure model factual beliefs and update incorrect beliefs in models . they propose a new visualization tool that shows relationships between stored model beliefs .
Outcome:	The proposed methods improve models' consistency and accuracy, the authors show . their methods outperform existing methods in more difficult settings, the paper shows .

Copied to clipboard

Yucheng Huang, Wenqiang Liu, Xianli Zhang, Jun Lang, Tieliang Gong, Chen Li

Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization (2020.findings-emnlp)

Challenge:	Existing methods to address the named entity recognition problem are limited and lack explicit optimization specific to the task.
Approach:	They propose a prototype-based representation alignment model for a cross-lingual named entity recognition task using labeled source language data.
Outcome:	The proposed model outperforms existing state-of-the-art methods in some challenging scenarios.

Copied to clipboard

Kunal Chawla, Diyi Yang

Partners in Crime: Multi-view Sequential Inference for Movie Understanding (D19-1)

Challenge:	Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences.
Approach:	They propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal.
Outcome:	The proposed model outperforms state-of-the-art models in terms of automated metrics and human judgement.

Copied to clipboard

Nikos Papasarantopoulos, Lea Frermann, Mirella Lapata, Shay B. Cohen

Challenge:	Existing multi-view learning approaches are tested in unsupervised setups, allowing for learning of representation for monolithic data points, not sequences.
Approach:	They propose a neural architecture paired with a novel objective for incremental inference that integrates multi-view information for sequence prediction problems.
Outcome:	The proposed model outperforms previous work and strong baselines on two crime cases and speaker type tagging tasks that contribute to movie understanding.

Pedagogically Aligned Objectives Create Reliable Automatic Cloze Tests (2024.naacl-long)

Copied to clipboard

Brian Ondov, Kush Attal, Dina Demner-Fushman

Exploring Question-Specific Rewards for Generating Deep Questions (2020.coling-main)

Challenge:	a recent study has focused on the cloze training objective of Masked Language Models . distractors must be distinct and incorrect, and can be biased if the test creator is testing two versions of a text .
Approach:	They propose a method that jointly optimizes sets of distractors from Masked Language Models.
Outcome:	The proposed method has stronger correlation with teacher-created comprehension tests than state-of-the-art neural method and is more internally consistent.

Copied to clipboard

Yuxi Xie, Liangming Pan, Dongzhe Wang, Min-Yen Kan, Yansong Feng

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning? (2024.lrec-main)

Challenge:	Recent question generation approaches use the sequence-to-sequence framework to optimize the log likelihood of ground-truth questions using teacher forcing.
Approach:	They propose to optimize for QG-specific objectives via reinforcement learning to improve question quality.
Outcome:	The proposed model improves the fluency, relevance, and answerability of generated questions.

Copied to clipboard

Shaoxiong Ji, Timothee Mickus, Vincent Segonne, Jörg Tiedemann

Multilinguality

Challenge:	Existing models that pretrain for cross-lingual tasks do not improve cross-linguistic learning.
Approach:	They propose to employ machine translation as a continued training objective to enhance language representation learning by bridging multilingual pretraining and cross-lingual applications.
Outcome:	The proposed model performance is compared with existing models and their latent representations.

Sentence Weighting for Neural Machine Translation Domain Adaptation (C18-1)

Copied to clipboard

Shiqi Zhang, Deyi Xiong

KidLM: Advancing Language Models for Children – Early Insights and Future Directions (2024.emnlp-main)

Challenge:	Neural machine translation (NMT) has achieved satisfactory performance on many language pairs with various advantages over statistical machine translation.
Approach:	They propose a new sentence weighting method for the domain adaptation of neural machine translation that uses a domain similarity metric to evaluate the relevance of sentences to the target domain.
Outcome:	The proposed method achieves significant improvement over baselines on Chinese-English TED task and synthetic training task with only synthetic training parallel data.

Copied to clipboard

Mir Tafseer Nayeem, Davood Rafiei

Framing Word Sense Disambiguation as a Multi-Label Problem for Model-Agnostic Knowledge Integration (2021.eacl-main)

Challenge:	Large language models have been shown to be effective in creating educational tools for children, yet there are significant challenges in maintaining key child-specific properties such as linguistic nuances, cognitive needs, and safety standards.
Approach:	They propose a user-centric data collection pipeline that involves gathering and validating a corpus specifically written for and sometimes by children.
Outcome:	The proposed model excels in understanding lower grade-level text, maintains safety by avoiding stereotypes, and captures children’s unique preferences.

Copied to clipboard

Simone Conia, Roberto Navigli

Information Extraction & Text Mining

Challenge:	Recent studies treat Word Sense Disambiguation (WSD) as a single-label classification problem, but gold data suggests that maximizing the probability of a singular sense is not the most suitable training objective for WSD.
Approach:	They propose to use Word Sense Disambiguation (WSD) as a multi-label classification problem in which multiple senses can be assigned to each target word.
Outcome:	The proposed method bears closer resemblance to how human annotators disambiguate text and can be extended to exploit structured knowledge from semantic networks.

Waste Not, Want Not; Recycled Gumbel Noise Improves Consistency in Natural Language Generation (2025.naacl-long)

Copied to clipboard

Damien De Mijolla, Hannan Saddiq, Kim Moore

Sentence-Level Agreement for Neural Machine Translation (P19-1)

Challenge:	Consistency in the output of language models can vary significantly in style, factual accuracy, and tone, even for similar inputs.
Approach:	They propose a decoding algorithm that enhances response consistency across different prompts with no degradation in response quality.
Outcome:	The proposed method outperforms standard sampling methods by 10% across semantic and stylistic consistency benchmarks.

Copied to clipboard

Mingming Yang, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Min Zhang, Tiejun Zhao

Multilinguality

Challenge:	Empirical results show that a sentence-level agreement module can significantly improve the performance of neural machine translation (NMT)
Approach:	They propose a sentence-level agreement module to minimize the difference between the representation of source and target sentences.
Outcome:	Empirical results show the proposed agreement module significantly improves translation performance.

Causal Distillation for Language Models (2022.naacl-main)

Copied to clipboard

Zhengxuan Wu, Atticus Geiger, Joshua Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, Noah Goodman

Data-Efficient French Language Modeling with CamemBERTa (2023.findings-acl)

Challenge:	Distillation efforts have led to language models that are more compact and efficient without serious drops in performance.
Approach:	They propose to augment distillation with a third objective that encourages the student model to imitate the causal dynamics of the teacher through a distillation interchange intervention training objective (DIITO).
Outcome:	The proposed method lowers perplexity on the WikiText-103M corpus and improves on the GLUE benchmark, SQuAD, and CoNLL-2003.

Copied to clipboard

Wissam Antoun, Benoît Sagot, Djamé Seddah

Adapting RNN Sequence Prediction Model to Multi-label Set Prediction (N19-1)

Challenge:	Recent advances in NLP have significantly improved the performance of language models on a variety of tasks.
Approach:	They introduce a French DeBERTa model that builds upon the DeBERTAV3 architecture and training objective and evaluate its performance on a variety of French downstream tasks and datasets.
Outcome:	The proposed model outperforms BERT-based models on most tasks given the same amount of training tokens and trained on 30% of its input tokens.

Copied to clipboard

Kechen Qin, Cheng Li, Virgil Pavlu, Javed Aslam

Information Extraction & Text Mining

Challenge:	Existing approaches to multi-label classification are based on pre-specifying the label order, or relating the sequence probability to the set probability in ad hoc ways.
Approach:	They propose a new training objective that maximizes this set probability and a prediction objective that finds the most probable set on a test document.
Outcome:	The proposed model outperforms existing methods on a set of labels for multi-label classification . the proposed model is based on 'set probability' and 'prediction objective'

Deep Inside-outside Recursive Autoencoder with All-span Objective (2020.coling-main)

Copied to clipboard

Ruyue Hong, Jiong Cai, Kewei Tu

A Contrastive Framework for Learning Sentence Representations from Pairwise and Triple-wise Perspective in Angular Space (2022.acl-long)

Challenge:	Existing neural approaches for constituency parsing are limited for low-resource languages and domains.
Approach:	They extend the training objective of DIORA by making use of all spans instead of only leaf-level spans.
Outcome:	The proposed model improves on two languages and provides better parsing accuracy than the original model.

Copied to clipboard

Yuhao Zhang, Hongji Zhu, Yongliang Wang, Nan Xu, Xiaobo Li, Binqiang Zhao

Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models (2021.acl-long)

Challenge:	Existing methods for learning sentence representations focus on constitution of positive and negative representation pairs and do not focus on training objective.
Approach:	They propose a new method to learn sentence representations using BERT-like pre-trained models . they use a pairwise discriminating power and a model to model the entailment relation of triplet sentences .
Outcome:	The proposed method outperforms the previous state-of-the-art on diverse sentence related tasks.

Copied to clipboard

Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum

Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables (2021.findings-emnlp)

Challenge:	Autoregressive neural machine translation (NMT) uses a tractable likelihood computation and efficient sampling.
Approach:	They propose to use an energy-based model to mimic the behavior of the task measure and use it to train an energy based re-ranking algorithm.
Outcome:	The proposed model improves on the samples drawn from the NMT with a higher BLEU score than the experimental model and the energy-based re-ranking algorithm.

Copied to clipboard

Weizhi Wang, Zhirui Zhang, Yichao Du, Boxing Chen, Jun Xie, Weihua Luo

Training Language Models with Memory Augmentation (2022.emnlp-main)

Challenge:	Existing methods to achieve zero-shot translation suffer from spurious correlations between output language and language invariant semantics.
Approach:	They propose a method that denoizes the autoencoder objective based on pivot language into traditional training objective to improve translation accuracy on zero-shot directions.
Outcome:	The proposed method eliminates spurious correlations and outperforms state-of-the-art methods on two benchmark machine translation datasets.

Copied to clipboard

Zexuan Zhong, Tao Lei, Danqi Chen

Intermediate Self-supervised Learning for Machine Translation Quality Estimation (2020.coling-main)

Challenge:	Existing methods for training memory-augmented language models only introduce mem-ories at testing time or represent them using a separately trained encoder.
Approach:	They propose a training approach that directly takes in-batch examples as accessible memory and new methods for memory construction and data batching that are used for adapting to different sets of memories at testing time.
Outcome:	The proposed approach reduces perplexity from 18.70 to 15.37 on multiple language modeling and machine translation benchmarks.

Copied to clipboard

Raphael Rubino, Eiichiro Sumita

Multilinguality

Challenge:	Existing methods for machine translation quality estimation (QE) rely on annotated data.
Approach:	They propose a self-supervised learning task for machine translation (MT) that orients a pre-trained model towards the target task.
Outcome:	The proposed method outperforms existing methods on English-to-German and English- to-Russian translation directions and is comparable to existing models.

Beyond Rejection Sampling: Trajectory Fusion for Scaling Mathematical Reasoning (2026.findings-acl)

Copied to clipboard

Jie Deng, Hanshuang Tong, Jun Li, Shining Liang, Ning Wu, Hongzhi Li, Yutao Xie

PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs (2024.findings-acl)

Challenge:	Large language models (LLMs) fine-tuned using rejection sampling retain only correct reasoning trajectories . however, this paradigm treats supervision as a binary filter that systematically excludes teacher-generated errors, leaving a gap in how reasoning failures are modeled during training.
Approach:	They propose a fine-tuning strategy that reframes rejection sampling as a structured supervision construction process.
Outcome:	The proposed approach outperforms RFT on multiple math benchmarks while retaining only correct reasoning trajectories.

Copied to clipboard

Jiuzhou Han, Nigel Collier, Wray Buntine, Ehsan Shareghi

Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution (2020.emnlp-main)

Challenge:	Existing large language models are pre-trained on unstructured data, which leads to poor performance when dealing with structured data.
Approach:	They propose a framework to train large language models to act as verifier modules and to apply iterative corrections offline.
Outcome:	The proposed framework improves graph-based generative capability of large language models by iterating corrective instructions on three graph-derived datasets.

Copied to clipboard

Yordan Yordanov, Oana-Maria Camburu, Vid Kocijan, Thomas Lukasiewicz

A Closer Look at Parameter Contributions When Training Neural Language and Translation Models (2022.coling-1)

Challenge:	Pre-trained language models have been used to obtain state-of-the-art results on pronoun resolution.
Approach:	They propose to use pre-trained language models to train pronoun resolution models . they compare performance and seed-wise stability of four models that represent four objectives .
Outcome:	The proposed model performs best in-domain, while the other objectives perform poorly out-of-domain.

Copied to clipboard

Raúl Vázquez, Hande Celikkanat, Vinit Ravishankar, Mathias Creutz, Jörg Tiedemann

RRHF-V: Ranking Responses to Mitigate Hallucinations in Multimodal Large Language Models with Human Feedback (2025.coling-main)

Challenge:	Neural models and Transformers have been used for almost every NLP task . however, the intrinsic dynamics of the training procedure have not been studied in depth for highly complex network architectures.
Approach:	They analyze the learning dynamics of neural language and translation models using Loss Change Allocation indicator . they use a standard Transformer architecture to train a model with three learning objectives .
Outcome:	The proposed model is based on a standard model that is used for training tasks.

Copied to clipboard

Guoqing Chen, Fu Zhang, Jinghao Lin, Chenglong Lu, Jingwei Cheng

Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Training (2024.emnlp-main)

Challenge:	Existing methods to mitigate hallucinations generate erroneous or fabricated information.
Approach:	They propose a rank-response-based model that annotates pair-reponses and trains alignment algorithms to improve the correspondence between images and text.
Outcome:	The proposed model outperforms the DPO method and outperfies existing methods on two MLLMs of different sizes and four widely used benchmarks.

Copied to clipboard

Wenbo Li, Guohao Li, Zhibin Lan, Xue Xu, Wanru Zhuang, Jiachen Liu, Xinyan Xiao, Jinsong Su

Visual Data in NLP

Challenge:	Existing text-to-image models struggle to generate images with legible visual texts . current models lack support for Chinese texts, misspelling, and lack of diversity .
Approach:	They propose to empower backbone models to generate visual texts in Chinese and English . they propose to augment conventional training objective with glyph-aware training losses .
Outcome:	The proposed methods can generate visual texts in English and Chinese while maintaining image generation quality.

Paired Examples as Indirect Supervision in Latent Decision Models (2021.emnlp-main)

Copied to clipboard

Nitish Gupta, Sameer Singh, Matt Gardner, Dan Roth

Challenge:	a new method to learn compositional structured models is needed . end-task supervision provides only a weak indirect signal on values the latent decisions should take.
Approach:	They propose a way to leverage paired examples that provide stronger cues for learning latent decisions . they use a DROP dataset to acquire paired questions that provide strong cue signals .
Outcome:	The proposed approach improves compositional question answering on a DROP dataset.

Evaluation of Transfer Learning for Polish with a Text-to-Text Model (2022.lrec-1)

Copied to clipboard

Aleksandra Chrabrowa, Łukasz Dragan, Karol Grzegorczyk, Dariusz Kajtoch, Mikołaj Koszowski, Robert Mroczkowski, Piotr Rybak

SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation (2022.findings-emnlp)

Challenge:	Recent years have brought significant progress in natural language understanding (NLU) and natural language generation (NLG).
Approach:	They propose a benchmark for assessing the quality of text-to-text models for Polish . they evaluate the performance of plT5, mT5, Polish BART, and Polish GPT-2 .
Outcome:	The proposed model can be fine-tuned on various NLP tasks with a single training objective.

Copied to clipboard

Yueqian Wang, Chang Liu, Kai Chen, Xi Wang, Dongyan Zhao

Responsible & Trustworthy NLP

Challenge:	Large-scale language models with prompts have shown remarkable performance on few-shot learning.
Approach:	They propose an approach to improve SMAll language models’ few-SHot ability by training on intermediate tasks before prompt-based fine-tuning on downstream tasks.
Outcome:	The proposed model improves on sentence-pair and sentiment classification tasks by training on intermediate tasks before fine-tuning on downstream tasks.

Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy (2025.emnlp-main)

Copied to clipboard

Nikita Balagansky, Yaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koriagin, Daniil Gavrilov

Explainability & Interpretability in NLP

Challenge:	Sparse Autoencoders (SAEs) are powerful tools for interpreting neural networks . conventional SAEs are constrained by the fixed sparsity level chosen during training .
Approach:	They propose a training objective that trains a single SAE to optimise reconstructions across multiple sparsity levels simultaneously.
Outcome:	The proposed objective achieves Pareto-optimal trade-offs between sparsity and explained variance, outperforming traditional SAEs trained at individual sparsities.

Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translation (2022.findings-emnlp)

Copied to clipboard

Michalis Korakakis, Andreas Vlachos

Towards Semi-Supervised Learning for Deep Semantic Role Labeling (D18-1)

Challenge:	Autoregressive models trained with maximum likelihood estimation suffer from exposure bias, i.e. the discrepancy between ground-truth prefixes used during training and model-generated prefix at inference time.
Approach:	They propose to use Elastic Weight Consolidation to better balance mitigating exposure bias with retaining performance.
Outcome:	The proposed method significantly outperforms maximum likelihood estimation and scheduled sampling baselines on four translation datasets.

Copied to clipboard

Sanket Vaibhav Mehta, Jay Yoon Lee, Jaime Carbonell

Thematic Fit Bits: Annotation Quality and Quantity Interplay for Event Participant Representation (2022.lrec-1)

Challenge:	Existing methods for semantic role labeling require an immense amount of semantic-role corpora and are therefore not suitable for low-resource languages or domains.
Approach:	They propose a semi-supervised method that outperforms the state-of-the-art on SRL . method explicitly enforcs syntactic constraints by augmenting the training objective with a syntastic-inconsistency loss component.
Outcome:	The proposed method outperforms the state-of-the-art on limited SRL training corpora on CoNLL-2012 English section.

Copied to clipboard

Yuval Marton, Asad Sayeed

Improving Neural Conversational Models with Entropy-Based Data Filtering (P19-1)

Challenge:	linguistically machine-annotated large corpus requires a large burden of labeled data.
Approach:	They compare linguistically machine-annotated large corpus output with higher-quality taggers to model thematic fit using a high-performing neural approach.
Outcome:	The proposed model shows that quality improves with training size, but plateaus or declines with size.

Copied to clipboard

Richárd Csáky, Patrik Purgai, Gábor Recski

Dialogue Systems & Conversational Agents

Challenge:	Current neural network-based conversational models lack diversity and generate boring responses to open-ended utterances.
Approach:	They propose an unsupervised method of filtering dialog datasets by removing generic utterances from training data using an entropy-based approach that does not require human supervision.
Outcome:	The proposed method improves dialog quality as chatbots learn to output more diverse responses to open-ended utterances.

PolCLIP: A Unified Image-Text Word Sense Disambiguation Model via Generating Multimodal Complementary Representations (2024.acl-long)

Copied to clipboard

Qihao Yang, Yong Li, Xuelin Wang, Fu Lee Wang, Tianyong Hao

Visual Data in NLP

Challenge:	Existing models for word sense disambiguation lack images or senses in textual and visual datasets.
Approach:	They propose a unified image-text WSD model that uses image-sense complementarity to generate visual representations for word senses and a disambiguation-oriented image-sensor dataset to provide implicit textual representations.
Outcome:	The proposed model achieves 2.53% F1-score increase over state-of-the-art models on Textual-WSD and 2.22% HR@1 improvement on Visual-WSS.

ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning (2026.acl-long)

Copied to clipboard

Lingxiao Tang, He Ye, Zhaoyang Chu, Muyang Ye, Zhongxin Liu, Xiaoxue Ren, Lingfeng Bao

Explainability & Interpretability in NLP

Challenge:	Existing methods for code execution reasoning are limited by the difficulty of the training data.
Approach:	They propose a model that uses reinforcement learning to reward correct answers from execution traces.
Outcome:	The proposed model improves pass@1 by up to 5.9% on code generation tasks over strong baselines.

Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective (2024.acl-long)

Copied to clipboard

Zihao Yue, Liang Zhang, Qin Jin

Diversifying Dialogue Generation with Non-Conversational Text (2020.acl-main)

Challenge:	Extensive research has shed light on the origins of multimodal hallucinations, including the inability of vision encoders to represent finegrained visual details and model reliance on inherent parametric knowledge such as language priors and statistical biases.
Approach:	They propose to use EOS to terminate generation of large multimodal models by comparing the generated text with the image to mitigate multimodal hallucinations.
Outcome:	The proposed method significantly improves the hallucination performance of Large Multimodal Models without additional data or knowledge.

Copied to clipboard

Hui Su, Xiaoyu Shen, Sanqiang Zhao, Zhou Xiao, Pengwei Hu, Randy Zhong, Cheng Niu, Jie Zhou

Dialogue Systems & Conversational Agents

Challenge:	Neural network-based sequence-to-sequence models suffer from low diversity in open-domain dialogue generation.
Approach:	They propose a way to diversify dialogue generation by leveraging non-conversational text . they collect large-scale corpus from forum comments, idioms and book snippets .
Outcome:	The proposed model produces significantly more diverse responses without sacrificing relevance with context.

Learning with Rejection for Abstractive Text Summarization (2022.emnlp-main)

Copied to clipboard

Meng Cao, Yue Dong, Jingyi He, Jackie Chi Kit Cheung

Incorporating Label Dependencies in Multilabel Stance Detection (D19-1)

Challenge:	Existing abstractive summarization systems produce non-factual summaries due to noise in the training dataset.
Approach:	They propose a training objective for abstractive summarization based on rejection learning that learns whether or not to reject potentially noisy tokens.
Outcome:	The proposed method significantly improves the factuality of generated summaries in automatic and human evaluations when compared to baseline models.

Copied to clipboard

William Ferreira, Andreas Vlachos

Sentiment Analysis

Challenge:	Stance detection is a well-studied task in social media, but previous work focused on multiclass versions of the problem where labels are mutually exclusive.
Approach:	They propose a method that explicitly incorporates label dependencies in the training objective and reduces multilabel to multiclass learning.
Outcome:	The proposed method improves on two out of three datasets and reduces multilabel to multiclass learning.

Contrastive Domain Adaptation for Question Answering using Limited Text Corpora (2021.emnlp-main)

Copied to clipboard

Zhenrui Yue, Bernhard Kratzwald, Stefan Feuerriegel

Low-Resource NLP

Challenge:	Existing question generation methods rely on large amounts of synthetically generated datasets and costly computational resources.
Approach:	They propose a framework for domain adaptation that combines question generation and domain-invariant learning to answer out-of-domain questions in settings with limited text corpora.
Outcome:	The proposed framework improves on state-of-the-art questions in a domain with limited text corpora.

GRI: Graph-based Relative Isomorphism of Word Embedding Spaces (2023.findings-emnlp)

Copied to clipboard

Muhammad Ali, Yan Hu, Jianbin Qin, Di Wang

Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor (2024.acl-long)

Challenge:	Existing attempts to control relative isomorphism of different spaces fail to consider lexical variations of semantically similar words . Existing methods for building bilingual dictionaries rely on geometric similarity of individual spaces .
Approach:	They propose a method that incorporates the impact of lexical variations of semantically similar words into the training objective.
Outcome:	The proposed method outperforms existing research by improving the average P@1 by 63.6%.

Copied to clipboard

Sangwon Yu, Changmin Lee, Hojin Lee, Sungroh Yoon

StoryTrans: Non-Parallel Story Author-Style Transfer with Discourse Representations and Content Enhancing (2023.acl-long)

Challenge:	Existing methods to control text generation are inapplicable to black-box models or suffer a trade-off between control and fluency.
Approach:	They propose a new approach to control text generation that modifies context at the token level during the generation process of a backbone language model and guides subsequent text to naturally include the target attributes.
Outcome:	The proposed method can regulate the attributes of the generated text while utilizing the capability of the backbone large language models.

Copied to clipboard

Xuekai Zhu, Jian Guan, Minlie Huang, Juan Liu

FLEXITOKENS: Flexible Tokenization for Evolving Language Models (2026.findings-acl)

Challenge:	Existing studies on text style transfer neglect long style transfer at the discourse level.
Approach:	They propose a model that transfers text style into target styles with learnable style embeddings . they use a mask-and-fill framework to explicitly fuse style-specific keywords into generation .
Outcome:	The proposed model outperforms baselines in style transfer and content preservation.

Copied to clipboard

Abraham Toluwase Owodunni, Orevaoghene Ahia, Sachin Kumar

Domain Adaptation via Prompt Learning for Alzheimer’s Detection (2024.findings-emnlp)

Challenge:	Widely used subword tokenizers overfragment sequences in unseen domains, languages, and scripts . inefficient tokenizer models can cause overfragments in out-of-distribution domains if not trained properly .
Approach:	They propose a byte-level LM with learnable tokenizers to make tokenization adaptive . they propose 'flexitoken' which enables significantly greater flexibility during adaptation .
Outcome:	The proposed method significantly reduces token overfragmentation and improves on multilingual benchmarks and domains.

Copied to clipboard

Shahla Farzana, Natalie Parde

Training Bilingual LMs with Data Constraints in the Targeted Language (2025.findings-acl)

Challenge:	Existing attempts to fine-tune pre-trained language models for AD detection are limited by the small and disparate corpora of spoken language.
Approach:	They propose to use domain-adaptive prompt fine-tuning to optimize for AD detection by using AD classification loss as the training objective and spoken language corpora from a variety of language tasks.
Outcome:	The proposed approach achieves the highest detection rate with the highest-performing prompting paradigm and the domain-adaptive prompt fine-tuning paradigm.

Copied to clipboard

Skyler Seto, Maartje Ter Hoeve, Richard He Bai, Natalie Schluter, David Grangier

EpiCaR: Knowing What You Don’t Know Matters for Better Reasoning in LLMs (2026.acl-long)

Challenge:	a large number of languages have insufficient data for pretraining, but most non-English models are trained on scrapes of the web.
Approach:	They propose to use data from an auxiliary language to boost model performance . they quantify the performance gap between training with data in a data-rich auxiliary and training in the target language .
Outcome:	The proposed method boosts model performance in a target language with insufficient data . it also explores the benefits of translation systems and the limitations of model scaling when data is limited.

Copied to clipboard

Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim

HMoE: Heterogeneous Mixture of Experts for Language Modeling (2025.emnlp-main)

Challenge:	Existing approaches to improving reasoning abilities of large language models incur a significant calibration cost.
Approach:	They propose an epistemic learning problem that integrates reasoning and calibration into an iterative supervised training framework.
Outcome:	The proposed method achieves Pareto-superiority over standard baselines in accuracy and calibration.

Copied to clipboard

An Wang, Xingwu Sun, Ruobing Xie, Shuaipeng Li, Jiaqi Zhu, Zhen Yang, Pinxue Zhao, Weidong Han, Zhanhui Kang, Di Wang, Naoaki Okazaki, Cheng-zhong Xu

Don’t Half-listen: Capturing Key-part Information in Continual Instruction Tuning (2025.acl-long)

Challenge:	Mixture of Experts (MoE) models use homogeneous experts with diverse capacities, resulting in a lack of expert specialization and parameter utilization.
Approach:	They propose a framework where experts differ in size and possess diverse capacities . they propose HMoE to encourage frequent activation of smaller experts .
Outcome:	The proposed framework outperforms homogeneous homogenous MoE models on evaluation benchmarks and achieves lower loss rate with fewer activated parameters.

Copied to clipboard

Yongquan He, Wenyuan Zhang, Xuancheng Huang, Peng Zhang, Lingxun Meng, Xiang Zhou, Ke Zeng, Xunliang Cai

Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning (2025.acl-long)

Challenge:	Existing methods to improve instruction tuning for large language models may cause catastrophic forgetting (CF) CF is a problem where previously learned abilities are degraded .
Approach:	They propose a continual instruction tuning method that uses key-part information gain to replay data and refine training objective.
Outcome:	The proposed method achieves superior performance on both seen and held-out tasks.

Copied to clipboard

Lei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan, Xiachong Feng, Yuxuan Gu, Yangfan Ye, Liang Zhao, Weihong Zhong, Baoxin Wang, Dayong Wu, Guoping Hu, Lingpeng Kong, Tong Xiao, Ting Liu, Bing Qin

Representation Degeneration Problem in Prompt-based Models for Natural Language Understanding (2024.lrec-main)

Challenge:	Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage.
Approach:	They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.
Outcome:	The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.

Copied to clipboard

Qingyan Zhao, Ruifang He, Jinpeng Zhang, Chang Liu, Bo Wang

VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervision (2026.acl-long)

Challenge:	Prompt-based fine-tuning (PF) models have shown improved performance on few-shot natural language understanding benchmarks.
Approach:	They propose a framework to alleviate anisotropy in the embedding space by aligning with pre-trained language models' training objective.
Outcome:	The proposed method outperforms mainstream methods on many NLU benchmarks.

Copied to clipboard

Xuan Gong, Senmiao Wang, Hanbo Huang, Ruoyu Sun, Shiyu Liang

Can Hallucination Correction Improve Video-Language Alignment? (2025.findings-acl)

Challenge:	Empirical evaluations demonstrate that VCORE achieves the strongest overall average performance, with especially clear gains on lower-capacity models.
Approach:	They propose a framework that reformulates supervision as a constrained optimization problem.
Outcome:	Empirical evaluations show that VCORE achieves the strongest overall average performance, with especially clear gains on lower-capacity models.

Copied to clipboard

Lingjun Zhao, Mingyang Xie, Paola Cascante-Bonilla, Hal Daumé Iii, Kwonjoon Lee

Logit Space Constrained Fine-Tuning for Mitigating Hallucinations in LLM-Based Recommender Systems (2025.emnlp-main)

Challenge:	Existing work on hallucination correction for large vision-language models focuses on mitigating hallucisations, but a new approach is needed to improve video-language alignment.
Approach:	They propose a self-training framework learning to correct hallucinations in descriptions that do not align with the video content.
Outcome:	The proposed framework improves video-language alignment by identifying and correcting inconsistencies in descriptions that do not align with the video content.

Copied to clipboard

Jianfeng Deng, Qingfeng Chen, Debo Cheng, Jiuyong Li, Lin Liu

Subtle Errors in Reasoning: Preference Learning via Error-injected Self-editing (2025.acl-long)

Challenge:	Existing LLM-based recommender systems rely on standard fine-tuning methodologies, often ignoring hallucination issues during the fine-uning process.
Approach:	They propose a logit space constraint-based fine-tuning framework to mitigate hallucination in LLM-based recommenders by incorporating Kullback–Leibler divergence into the training objective.
Outcome:	Experiments on two recommendation models with distinct LLM backbones and four real-world datasets show that LCFT reduces hallucination and enhances recommendation performance.

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Wenjun Hou, Yi Cheng, Chak Tou Leong, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation (2026.acl-long)

Challenge:	Existing studies to improve mathematical ability typically involve applying preference learning to step-wise solution pairs, but they overlook critical subtle errors.
Approach:	They propose a preference learning framework that injects predefined subtle errors into pivotal tokens to construct hard pairs for error mitigation.
Outcome:	Extensive experiments show that the proposed framework improves on Qwen2-7B-Instruct and MATH with 4.5K training samples.

Copied to clipboard

Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi, Linh Ngo Van, Nguyen Thi Ngoc Diep, Thien Huu Nguyen, Trung Le